There are a bunch of (minor) issues with the python code that could be improved (unused imports, non-pythonic code like x == None rather than x is None etc), but the scripts I ran locally seemed to work ok (qualifier: I'm not completely sure what you are trying to achieve with the files in this PR, and haven't examined everything in detail). I'd be happy to resolve these issues, if that's helpful? I commented on a few issues but this list isn't exhaustive.

Some questions:

1. is there anything in the CI that tests these scripts?
2. if not, then how would one go about testing that they work as intended (other than just running them and attempting to make a release)?

There are a bunch of (minor) issues with the python code that could be improved (unused imports, non-pythonic code like x == None rather than x is None etc), but the scripts I ran locally seemed to work ok (qualifier: I'm not completely sure what you are trying to achieve with the files in this PR, and haven't examined everything in detail). I'd be happy to resolve these issues, if that's helpful? I commented on a few issues but this list isn't exhaustive.

Sure, go ahead.

Some questions:

1. is there anything in the CI that tests these scripts?

Not for the scripts generating release notes and creating the stable branch.

But of course the scripts that actually make the release are tested, which is good because I touched them a little bit, too, and testing them manually is quite annoying.

2. if not, then how would one go about testing that they work as intended (other than just running them and attempting to make a release)?

So far I just didn't think it's worth the effort to try to automate testing for these. But if someone else were to spend efforts on it, of course I wouldn't mind... In each case the only somewhat OK way to test them I can think of is to use some fixed pre-defined set of inputs for them, and store a reference output in the repository, and compare against that.

• for generate_package_release_notes.py one could e.g. run generate_package_release_notes.py 4.12.1 4.12.2 and verify the output doesn't change
• for release_notes.py somewhat similar, but I can't see how to do it w/o adding something to the script: specifically, I think one would have to add a command line argument that specifies an upper bound for when the issues were created (or possibly an upper bound on the issue number, or whatever); we wouldn't use that bound in actual usage, but for tests it would allow running, say ./release_notes 4.12.0 --before=SOMEDATE and ./release_notes 4.12.1 --before=SOMEDATE with hopefully stable output (one should test both a "minor" and a "major" release as they have slightly different queries)
• for create_stable_branch.py I guess one would have to clone gap with full history (including at least master and stable-4.12; then git reset --hard master SOME_FIXED_COMMIT, say just before stable-4.12 was branched off; then add a way to disable the git pull --ff-only in the script; then ensure the 4.13dev tag and stable-4.12 branch are not there (just git branch -D stable-4.12) and run ./create_stable_branch.py 13 -- then verify that git diff master SHA-OF-PUBLIC-v4.13dev-TAG and git diff stable-4.12 SHA-OF-PUBLIC-v4.12.0-TAG report no diffs. And have sensible commit messages.

But again, I don't think it's that important to have CI tests for them: I tested them extensively while preparing this PR. Otherwise they are usually never changed. Of course they are also rarely used (just a few times per year for releases) and if they break it's holding up a release, but the risk is low overall, typically it would be some URL in the package distro changing (not that they should change, but that's one of the few external things that could change w/o anyone touching the scripts).

BTW ideally also generate_release_notes.py would be merged into release_notes.py to resolve #5004. The main "blockers" there are:

• generate_release_notes.py currently needs to be given the previous GAP release. Of course if we try to release 4.12.3 it is trivial to compute that 4.12.2 was the previous release. But for 4.13.0 it requires a little more work -- e.g. we may assume that all public tags have been fetched check for the presence of a tag v4.12.0, v4.12.1, v4.12.2, ... until it find one that does not exist (here v4.12.3) and then assumes that v4.12.2 is the last release.
• for testing it would be nice if I could give it a version that's not yet been tagged in the package distro (so "4.13.0" right now) and it would automatically detect this and instead of trying to fetch JSON files for v4.13.0, instead would fetch them for latest. Or maybe instead of "detecting" this (how exactly would it do that anyway?) one could just add an optional command line argument to force this (perhaps just as simple as ./release_notes.py 4.13.0 latest?)

Thanks @fingolfin I've opened a PR into your fork (and this branch) with some cleanups. I think there's at least one bug that still needs to be resolved:

fingolfin#116

@james-d-mitchell how about we merge this PR, and you add your changes in a fresh PR to this repo? That might be easier to manage and increases visibility for others :-)

Sure thing @fingolfin sounds good